Skip to content

✨ Adds filtering for Service Listing in Catalog's RPC API #7537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
Apr 22, 2025
Merged

✨ Adds filtering for Service Listing in Catalog's RPC API #7537

merged 20 commits into from
Apr 22, 2025

Conversation

@pcrespov
Copy link
Member

@pcrespov pcrespov commented Apr 15, 2025 •
edited
Loading

What do these changes do?

Introduces a filters: ServiceListFilters argument in the listing functions of the catalog RPC API to enable service filtering.

  • catalog rpc-client

    • Added filters: ServiceListFilters to list_services_paginated and list_my_service_history_paginated

  • catalog simcore-service

    • Controller layer (api.rpc)
      • Accepts filters: ServiceListFilters in listing endpoints
      • Maps ServiceListFilters to domain model ServiceFiltersDB
      • Updates RPC server mocks (pytest_simcore.helpers.catalog_rpc_service)
    • Service layer (service.catalog_services)
      • Handles filter: ServiceFiltersDB in listing logic
    • Repository layer (repository.services)
      • Accepts filter: ServiceFiltersDB
      • Applies filters using apply_services_filters in SQLAlchemy queries

Related issue/s

How to test

cd services/catalog
make install-dev
pytest -vv tests/unit/with_dbs/test_api_rpc.py
pytest -vv tests/unit/with_dbs/test_repositories.py

Dev-ops

@pcrespov pcrespov added a:services-library issues on packages/service-libs a:catalog catalog service labels Apr 15, 2025
@pcrespov pcrespov added this to the Pauwel Kwak milestone Apr 15, 2025
@pcrespov pcrespov self-assigned this Apr 15, 2025
@codecov
Copy link

codecov bot commented Apr 15, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 78.57143% with 18 lines in your changes missing coverage. Please review.

Project coverage is 87.65%. Comparing base (e42e1a3) to head (1945d04).
Report is 1 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #7537      +/-   ##
==========================================
+ Coverage   87.39%   87.65%   +0.25%     
==========================================
  Files        1700     1740      +40     
  Lines       65953    67458    +1505     
  Branches     1146     1121      -25     
==========================================
+ Hits        57642    59130    +1488     
- Misses       7990     8020      +30     
+ Partials      321      308      -13
Flag Coverage Δ
integrationtests 65.22% <100.00%> (+0.01%) ⬆️
unittests 86.83% <78.57%> (+0.27%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 93.91% <ø> (ø)
pkg_dask_task_models_library 97.09% <ø> (ø)
pkg_models_library 92.72% <100.00%> (+0.71%) ⬆️
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.18% <ø> (ø)
pkg_service_integration 69.98% <100.00%> (-0.06%) ⬇️
pkg_service_library 73.11% <0.00%> (+0.14%) ⬆️
pkg_settings_library 90.84% <ø> (ø)
pkg_simcore_sdk 85.40% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 91.23% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.46% <91.42%> (∅)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.29% <ø> (ø)
datcore_adapter 98.12% <ø> (ø)
director 76.78% <ø> (ø)
director_v2 91.30% <ø> (+0.02%) ⬆️
dynamic_scheduler 97.40% <ø> (ø)
dynamic_sidecar 90.11% <ø> (ø)
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.12% <ø> (-0.11%) ⬇️
storage 87.50% <ø> (+0.10%) ⬆️
webclient ∅ <ø> (∅)
webserver 86.04% <100.00%> (+0.07%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e42e1a3...1945d04. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pcrespov pcrespov force-pushed the is7532/catalog-rpc-filter branch from 3b3317c to fbc4d42 Compare April 17, 2025 08:14
@pcrespov pcrespov changed the title WIP: Is7532/catalog rpc filter ✨ Add Support for Service List Filtering in Catalog RPC API Apr 17, 2025
@pcrespov pcrespov changed the title ✨ Add Support for Service List Filtering in Catalog RPC API ✨ Adds filtering for Service Listing in Catalog's RPC API Apr 17, 2025
@pcrespov pcrespov force-pushed the is7532/catalog-rpc-filter branch from 5131891 to ad692f2 Compare April 17, 2025 12:42
@pcrespov pcrespov marked this pull request as ready for review April 17, 2025 13:00
Copilot AI reviewed Apr 17, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

services/catalog/tests/unit/with_dbs/test_repositories.py:556

  • [nitpick] Consider seeding the random number generator in _create_fake_release_versions to ensure deterministic fake version generation across test runs, which can improve reproducibility.
def _create_fake_release_versions(num_versions: int) -> set[str]:

@pcrespov pcrespov enabled auto-merge (squash) April 17, 2025 13:01
@pcrespov pcrespov force-pushed the is7532/catalog-rpc-filter branch from aa64014 to ee7c587 Compare April 17, 2025 15:16
sanderegg
sanderegg approved these changes Apr 21, 2025
View reviewed changes
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

bisgaard-itis
bisgaard-itis approved these changes Apr 22, 2025
View reviewed changes
Copy link
Contributor

@bisgaard-itis bisgaard-itis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for adding these so quickly.

GitHK
GitHK approved these changes Apr 22, 2025
View reviewed changes
@pcrespov pcrespov force-pushed the is7532/catalog-rpc-filter branch from ee9a679 to 5a177bd Compare April 22, 2025 08:39
pcrespov added 15 commits April 22, 2025 11:25
@pcrespov
✨ Refactor service key prefixes and enhance service type filtering in…
f2740fa 
… SQL queries
@pcrespov
✨ Add service key prefixes and refactor regex patterns for dynamic an…
ba0c3f6 
…d computational services
@pcrespov
✨ Refactor service filters application and enhance service key prefix…
805253c 
… handling in SQL queries
@pcrespov
fixes tests
9b3627e
@pcrespov
✨ Add service key formats for dynamic, computational, and frontend se…
4babc5b 
…rvices
@pcrespov
✨ Refactor service key handling by consolidating key prefixes and reg…
65ad67e 
…ex patterns for improved maintainability
@pcrespov
minor
281b939
@pcrespov
fixes tests
cabace7
@pcrespov
fixes tests
778ca3c
@pcrespov
@sanderegg review: relative imports, doc, private
aaa5c67
@pcrespov
@GitHK review: types
27debde
@pcrespov
@bisgaard-itis review: filters
a713ee8
@pcrespov
@sanderegg review: services constants
4b30eb5
@pcrespov
minor
0e0a0af
@pcrespov
fixes mypy
1945d04
@pcrespov pcrespov force-pushed the is7532/catalog-rpc-filter branch from f4db708 to 1945d04 Compare April 22, 2025 09:26
@sonarqubecloud
Copy link

sonarqubecloud bot commented Apr 22, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@pcrespov pcrespov merged commit e36b197 into ITISFoundation:master Apr 22, 2025
93 of 94 checks passed
@pcrespov pcrespov deleted the is7532/catalog-rpc-filter branch April 22, 2025 14:45
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request May 8, 2025
Closed
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@giancarloromeo giancarloromeo giancarloromeo approved these changes

@GitHK GitHK GitHK approved these changes

@sanderegg sanderegg sanderegg approved these changes

@matusdrobuliak66 matusdrobuliak66 matusdrobuliak66 approved these changes

@bisgaard-itis bisgaard-itis bisgaard-itis approved these changes

Assignees

@pcrespov pcrespov

Labels

a:catalog catalog service a:services-library issues on packages/service-libs

Projects

None yet

Milestone

Pauwel Kwak

Development

Successfully merging this pull request may close these issues.

Support filtering of dynamic/computational services in catalog rpc client

6 participants

@pcrespov @giancarloromeo @GitHK @sanderegg @matusdrobuliak66 @bisgaard-itis